Aspects of the exemplary embodiments disclosed herein relate to a system and method for the assessment of quality of photographic images and find particular application in an automated system for prediction of image quality based not only on aesthetic features but also on content features.
Digital photographic images are produced by professional photographers and amateurs in increasing numbers. Such images may be made accessible through a public website where they can be rated for quality and other characteristics by viewers of the website.
There has been considerable effort in the field of image quality assessment to design quality metrics that can predict the perceived image quality automatically. See, for example, Z. Wang, et al., The handbook of video databases: design and applications, Chapter 41, pp. 1041-1078, CRC press, 2003. One objective has been to extract descriptors from the digital image with a good correlation with human preference. See, H. Sheikh, et al., “A statistical evaluation of recent full reference image quality assessment algorithms,” IEEE Transactions on Image Processing, 15(11):3440-3451, November 2006. The presence or absence of specific signal level degradations such as random or structured noise (e.g., salt and pepper noise, jpeg artifacts, ringing) and blur were often used in the past to define the quality of a photographic image. However, high definition digital sensors are now readily available which allow photographers to overcome such degradations. Image quality assessment has more recently focused on the assessment of higher level features that go beyond low level image qualities. See, R. Datta, et al., “Studying aesthetics in photographic images using a computational approach,” ECCV (3), pp. 288-301, 2006 (hereinafter, “Datta 2006”); R. Datta, et al., “Learning the consensus on visual quality for next-generation image management,” MULTIMEDIA '07: Proc. 15th Intern'l Conf. on Multimedia, pp. 533-536, 2007 (hereinafter, “Datta 2007”); and R. Datta, et al., “Algorithmic inferencing of aesthetics and emotion in natural images: An exposition,” 15th IEEE Intern'l Conf. on Image Processing, pp. 105-108, October 2008.
The features which relate to image quality are often referred to as aesthetic features, because they are designed for capturing specific visual elements such as color combinations, composition, framing, and the like which are not directly related to the content of the image but which have an impact on the perceived quality of the image.
Despite the proliferation of annotated image data available through social networks, photo sharing websites, and the like, which could be used as training data, challenges for high-level quality assessment still remain. First, such data is often annotated with an intrinsic noise. When dealing with human preference, unanimous consensus is rare. Instead, general trends with varying proportions of outliers are often observed. While the amount of data use to train an automated system could be increased, this does not always solve the problem of noise.
Another challenge concerns the design of features to capture human preference. The features currently in use do not always correlate well with human perception. In other words, they are not powerful enough to capture all the visual information that a viewer would use in assessing image quality.
There remains a need for a system and method which can improve image quality assessment.